Papers with learning procedure
Bilingual Lexicon Induction with Semi-supervision in Non-Isometric Embedding Spaces (P19-1)
Copied to clipboard
| Challenge: | Recent work on bilingual lexicon induction (BLI) relies on an assumption about the isometry of two embedding spaces. |
| Approach: | They propose a semi-supervised approach that relaxes the isometric assumption while leveraging limited aligned bilingual lexicons and a larger set of unaligned word embeddings. |
| Outcome: | The proposed method obtains state-of-the-art results on 15 of 18 language pairs on the MUSE dataset and does particularly well when the embedding spaces don’t appear isometric. |
SGL: Speaking the Graph Languages of Semantic Parsing via Multilingual Translation (2021.naacl-main)
Copied to clipboard
| Challenge: | Graph-based semantic parsing is one of the most promising general-purpose meaning representations . owing to this heterogeneity, most research focused on solutions specific to a given formalism . |
| Approach: | They propose a multilingual neural machine translation framework for Graph-based semantic parsing . they propose Graph2seq architecture that trains with an MNMT objective . |
| Outcome: | The proposed framework outperforms all competitors on cross-lingual parsing tasks. |
Dynamic Knowledge Distillation for Pre-trained Language Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods conduct knowledge distillation statically, e.g., student model aligns output distribution to teacher model on pre-defined training dataset. |
| Approach: | They propose a dynamic knowledge distillation that empowers the student to adjust the learning procedure according to its competency . they find it is promising and provide discussions on potential future directions towards more efficient methods . |
| Outcome: | The proposed method can boost student model performance while accelerating training . the proposed method reduces memory usage and accelerates model inference . |
AVAST: Attentive Variational State Tracker in a Reinforced Navigator (2022.aacl-main)
Copied to clipboard
| Challenge: | Recent advances in reinforcement learning have been proposed to deal with robotic navigation problems, especially vision-and-language navigation task. |
| Approach: | They propose a method to approximate belief state distribution for the construction of a reinforced navigator by using a variational approach to approximate the unseen environment. |
| Outcome: | The proposed method improves generalization to the unseen environment which is barely achieved by traditional deterministic state tracker. |
Mitigating Dataset Artifacts in Natural Language Inference Through Automatic Contextual Data Augmentation and Learning Optimization (2022.lrec-1)
Copied to clipboard
| Challenge: | In recent years, natural language inference has been an emerging research area . a new data augmentation technique is used to augment pre-trained language models . |
| Approach: | They propose to combine automatic contextual data augmentation with a learning procedure for natural language inference. |
| Outcome: | The proposed method outperforms baseline pre-trained language models on benchmark datasets and adversarial examples. |
D2U: Distance-to-Uniform Learning for Out-of-Scope Detection (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for out-of-scope (OOS) detection use classifier confidence score, but model cannot infer correctly. |
| Approach: | They propose a zero-shot post-processing step that exploits the classification confidence score and the shape of the entire output distribution. |
| Outcome: | The proposed method improves performance when there is no OOS training data and learning procedure when OOS data is available. |
Meta Fine-Tuning Neural Language Models for Multi-Domain Text Mining (2020.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained neural language models improve learning for various NLP tasks by fine-tuning them on task-specific training sets. |
| Approach: | They propose a meta-learning procedure to fine-tune neural language models on task-specific training sets. |
| Outcome: | The proposed procedure solves a group of similar NLP tasks on a text mining dataset. |
Data Annealing for Informal Language Understanding Tasks (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models that improve formal and informal language understanding tasks do not transfer to informal data directly. |
| Approach: | They propose a data annealing transfer learning procedure to bridge the performance gap on informal natural language understanding tasks. |
| Outcome: | The proposed procedure outperforms state-of-the-art models on three common tasks. |